Repository navigation
feature(CI): Run the live tests, raw data included, against staging on dispatch - #22
Merged
Merged
Conversation
…n dispatch The live tests were pinned to production, so the router could not point them at staging. They now honour APERIODIC_API_URL, and ci.yml takes a base_url on workflow_dispatch with the staging keys, which the router's data bump uses as a release gate. raw_live_test.go adds raw data end to end: coverage (summary and one symbol), the preview with no key, DEMO-KEY refused outside the preview, and, with APERIODIC_RAW_API_KEY, a paid download of the June 2025 trades file followed by a run that skips it. A dispatched run fails up front if a staging secret is missing, so a release cannot pass with the raw test skipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CkNoeMr2UCifeweNbR8R25
…e one The paid raw download read APERIODIC_RAW_API_KEY, so each environment needed a second key. It now uses APERIODIC_API_KEY like the other live tests, so that key (production, or staging when dispatched) has to be on the Prime + Raw plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CkNoeMr2UCifeweNbR8R25
Contributor
Author
|
Red on
Since this commit, the paid raw download uses the repo's existing To get green: upgrade the account behind Generated by Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The router's data bump runs the client SDK's tests against staging before every aperiodic.io release, but not the CLI's. dream-faster/unravel-router#884 now dispatches this repo's
ci.ymlas well. For that to work, the CLI needs a way to be pointed at staging, and it needs live tests for raw data.Changes
APERIODIC_API_URL.requireAPIKeyused to force production. It now calls a newuseLiveAPI(t): production unlessAPERIODIC_API_URLnames another API. The invalid-key tests use it too, so a staging run doesn't touch production.raw_live_test.go, end to end against the live API, with the June 2025 BTC perpetual files:raw coveragehas a summary row for each of the six datasets;raw coverage --dataset trades --exchange binance-futures --symbol …shows one row spanning June 2025;raw funding_rate --previewwith no key writes a Parquet file (PAR1at both ends) at the Hive path;DEMO-KEYoutside the preview exits 1 with a 401;APERIODIC_API_KEY:raw tradesdownloads the June file, and a second run reportsSkipping 1 already present. This is the one large transfer (the trades month, a few hundred MB).ci.ymltakesworkflow_dispatchwith abase_url. A dispatched run usesAPERIODIC_STAGING_API_KEYand the Cloudflare Access token. It fails up front if any of those three secrets is missing: a missing staging key would silently fall back to the production one, and missing Access credentials would fail every request. Push and PR runs still test production withAPERIODIC_API_KEY.One key per environment: the paid raw download uses the same
APERIODIC_API_KEYas every other live test, with no separate raw key. So the key in use (production, or staging when dispatched) must be on Prime + Raw. Until the production key is upgraded, the paid download fails on push and PR runs with a 403raw_not_in_plan.Needed before the next data bump
APERIODIC_API_KEYandAPERIODIC_STAGING_API_KEYto Prime + Raw. If the staging one is a seeded router e2e account whose tier the router's specs depend on (scripts/seed-staging-accounts.ts; e.g.e2e+primemust getraw_not_in_plan), point the secret ate2e+primerawinstead.APERIODIC_STAGING_API_KEY,CF_ACCESS_CLIENT_IDandCF_ACCESS_CLIENT_SECRETon this repo (or at org level): the same values aperiodic-io/client uses.Merge this before the router PR: until it lands,
ci.ymlonmainhas nobase_urlinput to dispatch.Verified
gofmt -l .is clean,go vet ./...passes, and the offline tests pass.APERIODIC_API_KEYset. It has not yet run against a real Prime + Raw key.actionlintonci.yml: clean.Spotted, not fixed here
Since unravel-router#879 (live in data-v1.3.13),
/metadata/rawno longer sendsbytesormissing. Soaperiodic raw coveragenow prints0 Bin SIZE and0in MISSING on every row. The new tests don't assert those columns.🤖 Generated with Claude Code
https://claude.ai/code/session_01CkNoeMr2UCifeweNbR8R25